1. Field of the Invention
The present invention relates to an image data coding method of coding the image data with a high efficiency, an apparatus for coding an image data, a restoring method of decoding the coded image and an apparatus for restoring the same and more particularly to an image data coding and decoding method of dividing the image data into a plurality of blocks to produce the coded data of all the images related to the respective blocks obtained by dividing the image data.
2. Description of the Related Art
In the field of a television conference/television telephone involving colored static images and colored moving image, the image data whose information capacity is much greater than typical numerical data are used, and more particularly, the data of the intermediate level of the image, or the colored image data requires high-speed and high quantity transmission. Therefore, it is necessary to process the coding of the values of the respective image elements with a high efficiency.
The adaptive discrete cosine transform method is known as an example of an image data coding method with a high efficiency.
This adaptive discrete cosine transform method (which is abbreviated as ADCT) divides the image into a plurality of blocks comprising, for example, 8.times.8 image elements, transforms the image signal of respective blocks into (two-dimensional DCT coefficients corresponding to the distributions of the space frequency by the two-dimensional discrete cosine transform), quantizes the two-dimensional DCT coefficient within a block obtained by the transform by using a threshold value suitable for vision, and then encodes the quantized coefficient by using a huffman table obtained statistically.
The conventional coding device for the static image using the ADCT encodes the respective images separately, even if the respective values between images do not change greatly because of a close interrelation between them, for example, in the case of a moving image which is considered as a group of static images.
FIG. 1 shows an example of a structure of the conventional coding apparatus for a static image.
A static image is divided into a plurality of blocks comprising 8.times.8 image elements, for example, and is inputted from the input terminal 11 and the respective image signals are sequentially restored in the input order to the block buffer 12.
The two-dimensional DCT unit 13 reads the respective image signals from block buffer 12 by transforming the image signal corresponding to a plurality of image elements in respective blocks into the two-dimensional DCT coefficient corresponding to the space frequency of the same number as the plurality of image elements by using the two-dimensional discrete cosine transform.
Linear quantization unit 14 quantizes the two-dimensional DCT coefficient (matrix) of respective blocks by using the quantizing threshold value (quantizing matrix) which is suitable for vision and obtained from visual experimentation. The linear quantization unit 14 reads the quantizing threshold value from a quantizing threshold unit.
Variable length coding unit 16 performs a variable length coding of the quantization coefficient obtained by quantizing unit 14 by using the huffman coding table 17 obtained statistically, and outputs from the output terminal 18 the coded data obtained from the variable length coding method.
In the variable length coding method, the quantizing coefficient (matrix) arranged in a two-dimensional manner is converted to a one-dimensional train of numerical values by using a scanning operation called zig-zag scanning and performs a variable length coding based on the difference between the head DCT coefficient of a block and the head DCT coefficient of the preceding block with regard to the DC component. The value of the valid coefficients whose value is not zero is combined with the length of run (Run-length of the invalid coefficient whose value is zero), thereby applying the variable length coding to respective blocks.
FIG. 2 shows a structure of a conventional restoring apparatus of the static image for restoring data coded by the above-recited variable length coding to the images.
The data coded by the coding apparatus for the static image is inputted from the input terminal 21 to variable length restoring unit 22 in units of blocks.
The variable length restoring unit 22 restores the quantizing coefficient of the DC component and AC components in the respective blocks using the huffman coding table 17 (huffman decoding table 27) which is used in the above variable length coding.
Dequantization unit 23 converts the decoded quantization coefficient of respective DC component and AC component into the two-dimensional DCT coefficients by using the quantized value 29 (quantized matrix).
Inverse DCT transforming unit 24 performs the two-dimensional DCT transform by using the two-dimensional DC coefficient of the respective blocks which are decoded and restored as the image elements.
The image signal of respective restored image elements is stored within the image memory 25 in units of blocks.
An operation of the image signal of respective image elements at image blocks into the image memory 25 is controlled by inside-block image element writing controlling unit 26 and controls the block address generating unit 27 and enables it to output an address of a block in which the image signal is to be written into the image memory 25 and performs a writing operation of the image signal of respective image elements of the present frame into the image memory 25 in units of blocks.
There is a method of coding a moving image which is subject to a continuous scene, i.e. a television conference or a television telephone system. Conventionally, the respective images are subject to the methods of coding the moving images.
FIG. 3 shows a block diagram of a basic part of the moving image coding apparatus by using an inter-frame prediction coding method of utilizing a relation between the image in reference frame and the image in the present frame.
The image of respective scenes of the moving image is inputted to difference image generating unit 32 through input terminal unit 31.
The reference image which is obtained by restoring the image in the preceding frame and is used as the image in the reference frame.
The difference generating unit 32 obtains the difference image between the reference images stored in an image memory 33 and the present image (the image in the present frame) inputted from the input terminal 31 and stores the difference image in block buffer 34. The difference image can be obtained for respective blocks comprising image elements of 8.times.8, for example. Therefore, the block buffer 34 stores the difference image i.e difference block image between the block in the reference frame of 8.times.8 image element, for example and the corresponding block in the present frame of 8.times.8 image elements with regard to the blocks in the same position.
The two-dimensional DCT transform 35 applies the two-dimensional DCT transform to the difference block image of 8.times.8 image elements stored in the block buffer 34 obtains the 8.times.8 DCT coefficients, and thereby outputs them to quantization unit 36.
Quantization unit 36 applies a linear quantization to 8.times.8 coefficients by using adequate threshold value applicable to an eyesight, thereby providing 8.times.8 quantized coefficients.
The 8.times.8 quantization coefficients are subject to zig-zag scanning and is outputted to variable length coding unit 37 in the order of the DC component, AC components with a low space frequency AC components with relatively high space frequency.
The variable coding unit 37 performs a variable length coding of the 8.times.8 input quantized coefficients by using huffman coding table 38 and outputs the data coded by the variable length coding from the output terminal 39.
The 8.times.8 quantizes coefficient matrix outputted from quantization unit 36 is inputted to the dequantization unit 39 and the dequantization unit 39 restores it into the two-dimensional DCT coefficient matrix.
Inverse two-dimensional DCT transform unit 40 performs the inverse two-dimensional DCT transform by using 8.times.8 two-dimensional coefficient and restores the above difference block image.
The adding unit 41 adds the restored difference block image to the preceding image of the block in the same position in the reference image as in the position in the present block image and again rewrites the image obtained by the addition into the image memory 33.
The above-recited sequential operation is applied to all the blocks of the image (the present image) of the new frame inputted from the input terminal 41, thereby enabling the coded data of the difference image regarding all the blocks of the present image to be outputted from the output terminal 39. As discussed above, the image data of the present frame is not coded as it is and a difference image between the reference frame and the present frame is coded, thereby decreasing the amount of the coding. The same image as restored from the image of the present frame, which is restored based on the coded data of the difference image on the image restoring side, is stored in the image memory 33 as the reference image to be used as the image of the reference frame when the image of the succeeding frame is inputted from the input terminal 31.
FIG. 4 shows a principle structure of a conventional restoring apparatus of the moving image for restoring the image of the coded data outputted from the conventional coding device for the moving image.
A variable length restoring unit 52 inputs the coded data obtained by applying a variable length coding to the difference image in units of block from the input terminal 51 and restores the coded data into quantization coefficients in units of blocks by using Huffman coding table 58, which is the same as the Huffman coding table 38 in the conventional coding apparatus for the moving image.
Dequantization unit 53 performs a dequantization of quantization coefficients of respective blocks by using the same matrix as used in the dequantization units 39 in the coding apparatus for the moving image, and restores the two-dimensional DCT coefficients of respective blocks. Inverse two-dimensional DCT transforming unit 54 restores the difference image of respective blocks by performing the inverse two-dimensional DCT conversion in units of blocks by using the two-dimensional DCT coefficient matrixes of respective blocks.
Adding unit 55 adds the decoded difference image of respective blocks to the image in the block in the same position in image memory 56 and the image obtained by this adding operation is rewritten in image memory 56.
A writing operation of a restored image into image memory 56 for respective blocks is controlled by intra-block image element write controlling unit 57. Intra-block image element write controlling unit 57 controls block address generating unit 59 and outputs an address of the particular block which is stored in the image memory and to which the difference image is added, to image memory 56 from block address generating unit 59, thereby writing the restoration image of respective frames in the image memory 56 in units of blocks.
The above sequential process is applied to the coded data of the difference image of all blocks of respective frames of the image and thus image memory 56 stores the restoration image of respective frames.
However, the above recited conventional technology has the following problems.
In the conventional coding technology of the static image, even respective images having small variation between continuous images are independently coded. Thus, even if the variation between the continuous images is very small such as images for a moving or a television telephone having a camera fixed, the compression ratio cannot be increased.
When the coding technology for the moving image as shown in FIG. 3 is used, the compression ratio can be raised owing to the difference coding method but it requires a larger scale of apparatuses. In the conventional coding apparatus for the moving image, the difference image between the present frame and the reference frame is calculated and the image of the reference frame to be used for obtaining the difference image is stored in the image memory 33. The variable length coding to the above ADCT is irreversible coding (information loss coding) and thus the error of the difference image from the original image becomes large when the number of accumulations exceeds a predetermined number and thus the quality of the image stored in the image memory 32 is decreased evidently. Therefore, it is necessary to refresh the image in an inter-frame mode at a predetermined number of frame intervals.
The inter-frame mode performs a variable length coding of the input image according to the ADCT in the similar manner to the conventional coding apparatus for the static image.
When the inverse two-dimensional DCT transform unit 40 used in the conventional decoding apparatus of the moving image is different from the two-dimensional DCT transform unit 54 used in the conventional coding apparatus for the moving image in respect of an operation or operation method, the errors generated in coding operations are stored and accumulated in the image memories 33 and 56, the noise caused by the accumulated errors becomes evident. Therefore, CCITT recommendation H-261 recommends that a refreshing operation should be performed at least once by using the above inter-frame mode during the period of 132 coding operations to solve the above problem of the accumulated errors generated due to a mis-match between the inverse two-dimensional DCT transform unit 40 and 54 in both apparatuses.
In the inter-frame mode, the image data of all blocks of one frame is coded as it is and the coded data of all blocks of one frame are transmitted to the receiving side. The frame in which such a coding is performed is called a refresh frame and such a coding is called refreshing. A quantity of the coding of the refresh frame in which the image data of all the blocks of one frame are coded as is, is greatly different from the quantity of coding of the ordinary frame in which only the difference image is coded. Namely, a quantity of the coding of the refresh frame is usually larger than that of the ordinary frames.
Therefore, when the image is transmitted by performing the data compression using such a high efficiency coding method as described above in television conference or a television telephone, the receiver of the coded data restores the frame after a longer time period passes upon receipt of the coded data of the refreshing frame than upon receipt of the coded data of an ordinary frame.
This difference in receiving time causes a receiver of the image to feel it naturally long.
When the image is transmitted at ten frames per second through an ISDN (Integrated Service Digital Network) with the transmission rate of 64 Kbps provided by a Japanese Telecommunication and Telephone Company, the amount of coded data of one frame should be less than 6400 bits.
Generally speaking, a quantity of coded data of one frame in the case of the ordinary frame can be compressed less than 6400 bits but the quantity of the coded data of one frame in the case of the refreshed frame is sometimes more than 6400 bits.
When a quantity of coded data per one frame exceeds the maximum quantity of coded data (which is 6400 bits in this case) of one frame which is originally assigned to one frame, the transmission processing of the refresh frame cannot be completed within the processing time assigned to the refresh frame and is extended into the processing time of the next frame.
In this case, the coding process of the next frame is not performed and the transmission process of the refresh frame will be performed. As a result, the frame which is not subject to the coding process is not transmitted to the receiving side and the monitor screen on the receiving side is displayed with one frame missing.
This example is shown in FIG. 5.
In FIG. 5, the third frame is a refresh frame and quantity of the coded data of the refresh frame exceeds 6400 bits. Therefore, the image of the third frame is not restored at time T3 when the image of the third frame should be restored on the monitor screen in the receiving side. An image of the third frame is restored at time T4, at which the image of the fourth frame should be restored. Thus, the image of the frames following the fourth frame are restored with one frame delay.
In the case of transferring the moving image, the coding side and decoding side have the coding buffers.
These code buffers are respectively provided in a succeeding stage of the variable length coding unit 37 and in a preceding stage of the variable length decoding unit 52 as shown in FIGS. 3 and 4. These code buffers have to store the data when the refreshing operation is performed, and thus the capacity of these code buffers depend on the maximum amount of the data of one frame, namely, the amount of data required for the refreshing operation. Accordingly, the decrease of the capacity of these code buffers greatly contribute to realization of a small size and inexpensive coding and decoding apparatus.
When the refreshing operation is performed, the image data of all the blocks of the refresh frame are coded without being subject to any process, as recited above and thus the amount of the coded data in the refreshing frame is greater than that in the ordinary frame which updates only the difference image. Therefore, there is a problem that a time spent to update the reference frame upon performing the refreshing operation becomes longer than that to update the reference frame upon coding the ordinary frame.
The variation between two consecutive frames in the moving image depends on the kind of the image. Therefore, upon performing the refreshing operation, the amount of the data is increased in the case of the image with a large movement and thus the refreshing operation sometimes cannot satisfy various conditions of the transmission line such as an information transmitting rate and the number of frames transmitted per unit time.
There is also a problem that the image appears strange as if it flashes at the time of the refreshing operation since a large amount of errors compared to the original image is accumulated in the image data of the reference frame before the refreshing operation is applied to the reference frame.